MDEV-10526: Add binary string support to bitwise operators#5190
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces support for binary-to-binary bitwise operations (AND, OR, XOR, NOT, and bitwise shifts) on binary strings by implementing new handler classes and adding corresponding error messages for mismatched operand sizes. The review feedback highlights several critical issues: a potential integer overflow/wrap-around in the left-shift index calculation on 32-bit systems, incorrect ASCII string conversion of numeric arguments in mixed-type bitwise operations, and potential undefined behavior when handling empty binary strings due to passing a null pointer to memset.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| size_t src_idx= i + byte_shift; | ||
| if (src_idx < len) | ||
| { |
There was a problem hiding this comment.
The calculation size_t src_idx = i + byte_shift is susceptible to integer overflow/wrap-around on 32-bit systems if len is very large (e.g., close to size_t max). If i + byte_shift overflows, src_idx wraps around and can evaluate to a value less than len, leading to incorrect data copying from wrapped-around indices instead of zero-filling. To prevent this, perform the bounds check using subtraction (byte_shift < len - i) before computing src_idx.
if (byte_shift < len - i)
{
size_t src_idx= i + byte_shift;| StringBuffer<128> b_buf; | ||
| String *b= item->arguments()[1]->val_str(&b_buf); |
There was a problem hiding this comment.
In mixed-type bitwise operations (e.g., binary_string & numeric), calling val_str on the numeric argument converts it to its decimal string representation (e.g., 255 becomes the 3-byte ASCII string "255" / 0x323535), rather than its binary representation (e.g., 0xFF). This causes mismatched length errors (e.g., VARBINARY(1) & 255 compares length 1 with length 3) or incorrect bitwise operations if the lengths happen to match. To align with standard bitwise behavior (and MySQL 8.0 compatibility), numeric arguments should be converted to a binary string representation of their integer value, padded or truncated to match the length of the other binary string operand.
| size_t len= a->length(); | ||
|
|
||
| if (to->realloc(len)) |
There was a problem hiding this comment.
When the input binary string a is empty (len == 0), calling to->realloc(0) can return a null pointer or do nothing. Subsequently, passing a null pointer to memset (e.g., memset(out_ptr, 0, len)) is technically undefined behavior in C/C++, even if the length is 0. Adding an early exit for len == 0 avoids this potential undefined behavior and improves efficiency by bypassing unnecessary allocation and loop overhead.
size_t len= a->length();
if (len == 0)
{
to->length(0);
to->set_charset(&my_charset_bin);
item->null_value= false;
return to;
}
if (to->realloc(len))fa5cd7c to
862a606
Compare
grooverdan
left a comment
There was a problem hiding this comment.
Should be off the main branch as a new feature.
from @abarkov
"Would be nice to add tests that uuid, inet6, inet4, geometry are not allowed for bit operations. Later we can probably implement bit operations for uuid, inet6, inet4. But to avoid compatibility problems we need to make sure they return error now."
221664d to
6b9dd9e
Compare
gkodinov
left a comment
There was a problem hiding this comment.
Thank you for your contribution! This is a preliminary review.
Please squash your commits to 1 per feature. Maybe keep the spider part in a separate commit and the rest of it in another? Just a suggestion. But there definitely should not be 13 commits.
| # ========================================================================= | ||
| CREATE TABLE t1 (a VARBINARY(4), b VARBINARY(4)); | ||
| INSERT INTO t1 VALUES (x'FFFF0000', x'FF00FF00'); | ||
| # Expected output: FF000000, FFFFFF00, 00FFFF00, 0000FFFF, FFFE0000, 7FFF8000 |
There was a problem hiding this comment.
Here and everywhere: you do not need this line. The expected result is already recorded.
| SELECT HEX(INET6_ATON('ffff:ffff::') & INET6_ATON('2001:db8::1')); | ||
| HEX(INET6_ATON('ffff:ffff::') & INET6_ATON('2001:db8::1')) | ||
| 20010DB8000000000000000000000000 | ||
| # Expected output: TBD - verify via --record |
There was a problem hiding this comment.
especially this needs to go away
| # ========================================================================= | ||
| CREATE TABLE t_null (a VARBINARY(4), b VARBINARY(4)); | ||
| INSERT INTO t_null VALUES (x'FFFF0000', NULL); | ||
| # Expected output: NULL, NULL, NULL, NULL, NULL, NULL |
There was a problem hiding this comment.
do you really need a table for that?
…ates
Extends all six scalar bitwise operators (&, |, ^, ~, <<, >>)
and aggregate functions (BIT_AND, BIT_OR, BIT_XOR) to work
byte-by-byte on BINARY/VARBINARY columns, returning VARBINARY
of the same length.
Previously all operators silently cast binary arguments to
BIGINT (64-bit), truncating values wider than 64 bits. This
broke operations on INET6 addresses, UUIDs, and any binary
column wider than 8 bytes.
Binary mode activates when both operands have a true non-hybrid
binary string type_handler (Type_handler_longstr with my_charset_bin
collation, excluding hex hybrids like 0xFF). Uses the existing
Item_handled_func pluggable Handler pattern, adding new
Handler_str subclasses per operator alongside existing
Handler_ulonglong handlers.
Aggregate functions gain a parallel String accumulator with
correct neutral elements (0xFF for AND, 0x00 for OR/XOR).
Window function support added via a dynamically-sized uint32
bit-counter array (same sliding-window technique as the
existing 64-bit bit_counters[], extended to arbitrary lengths).
Key fixes included:
- reset_field()/update_field() binary mode (crash fix:
raw int8store on string-typed temp table field caused SIGSEGV)
- return_type_handler() varies by max_length following the
blob_type_handler/type_handler_varchar/type_handler_string
pattern from Item_char_typecast_func_handler_fbt_to_binary
- DERIVATION_COERCIBLE (not IMPLICIT) for binary results
- MY_MAX() overflow pattern replaced with plain increment
in window function counters
- Mismatched lengths push warning (not hard error), escalating
to error under strict SQL mode via THD::raise_condition()
- Regressions in main.gis (geometry entering binary mode via
STRING_RESULT+my_charset_bin) and main.func_json (JSON
over-rejected by stricter check_arguments() override) fixed
- GEOMETRY and hex hybrids excluded from binary mode detection
New error codes:
ER_INVALID_BITWISE_OPERANDS_SIZE
ER_INVALID_BITWISE_AGGREGATE_OPERANDS_SIZE
Closes: MDEV-10526
Implements Spider direct_add() support for BIT_AND/BIT_OR/BIT_XOR, allowing each remote shard to compute its own partial aggregate merged locally, mirroring the existing SUM/COUNT/MIN/MAX pattern. BIT_AND/OR/XOR are associative and commutative so partial results from shards can be merged correctly via direct_add(). Also fixes a pre-existing Spider bug: row->val_int() uses atoi() which silently overflows for BIGINT UNSIGNED values above INT_MAX. Fixed for this code path using my_strtoll10() instead. Closes: MDEV-10526
934bb95 to
75cfdf6
Compare
gkodinov
left a comment
There was a problem hiding this comment.
LGTM. Please stand by for the final review.
| --echo # ========================================================================= | ||
|
|
||
| SELECT HEX(INET6_ATON('ffff:ffff::') & INET6_ATON('2001:db8::1')); | ||
|
|
|
I'm reviewing the spider bit |
|
|
||
| if (to->realloc(len)) | ||
| { | ||
| item->null_value= true; |
There was a problem hiding this comment.
we'd need to push a warning/error here.
my_error(ER_OUTOFMEMORY, MYF(0), len);
|
|
||
| String *val_str(Item_handled_func *item, String *to) const override | ||
| { | ||
| DBUG_ASSERT(item->fixed()); |
There was a problem hiding this comment.
Could be overkill, I can't see a way to call this wthout an arg, but just in case DBUG_ASSERT(arg_count ==2);
| String *a= item->arguments()[0]->val_str(&a_buf); | ||
| if (a == nullptr) | ||
| { | ||
| item->null_value= true; |
There was a problem hiding this comment.
normally this would just be null_value= true for the current object. Is null and invalid argument? Do we need to test if (item->arguments[0]->null_value) before val_str?
| const uchar *a_ptr= (const uchar *) a->ptr(); | ||
|
|
||
| if (shift_count_signed < 0 || (ulonglong)shift_count_signed >= (ulonglong)len * 8) | ||
| { |
There was a problem hiding this comment.
on style is ok to have a single line (only) in a if like condition without braces if its clear.
|
|
||
| if (to->realloc(len)) | ||
| { | ||
| item->null_value= true; |
There was a problem hiding this comment.
my_error(ER_OUTOFMEMORY, MYF(0), len);
| @@ -2257,10 +2370,123 @@ class Func_handler_shift_right_decimal_to_ulonglong: | |||
| }; | |||
|
|
|||
|
|
|||
| class Func_handler_shift_right_bin_to_bin: public Item_handled_func::Handler_str | |||
There was a problem hiding this comment.
There's a large amount of duplication of Func_hanlder_shift_left_bin_to_bin. Can it be refactored leaving just the ::val_str in each shift direction? Include a common argument handler called by val_str.
| item->max_length= item->arguments()[0]->max_length; | ||
| item->collation.set(&my_charset_bin, DERIVATION_COERCIBLE); | ||
| return false; | ||
| } |
There was a problem hiding this comment.
^ common shared with shift type handers. Can the commonality be in the same class.
| if (a->length() != b->length()) | ||
| { | ||
| THD *thd= current_thd; | ||
| push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, |
There was a problem hiding this comment.
Nice initialisation of thd only when needed. Do other null returns need similar warnings?
| { | ||
| if (!m_binary_bit_counters) | ||
| return; | ||
| if (m_str_value.alloc(m_binary_length)) |
| virtual void set_bits_from_counters()= 0; | ||
| bool m_binary_mode; |
There was a problem hiding this comment.
group member of same type together. Keeps the size of the structure smaller due to the way its packed.
| @@ -0,0 +1,154 @@ | |||
| --disable_warnings | |||
There was a problem hiding this comment.
Why disable warnings?
There was a problem hiding this comment.
This test is written in a similar style to some older tests, which is too complex and convoluted.
There's no need to have multiple mariadb servers to test this. Just one server with multiple remote tables should be sufficient.
Newer single-server tests are typically written like this:
--disable_query_log
--disable_result_log
--source ../../t/test_init.inc
--enable_result_log
--enable_query_log
--source include/have_innodb.inc
set spider_same_server_link= 1;
set global spider_same_server_link= 1;
evalp CREATE SERVER srv FOREIGN DATA WRAPPER mysql
OPTIONS (SOCKET "$MASTER_1_MYSOCK", DATABASE 'test',user 'root');
CREATE TABLE t1 (...) ENGINE=InnoDB;
CREATE TABLE t1_s (...)
ENGINE=SPIDER wrapper=mariadb REMOTE_SERVER=srv REMOTE_TABLE=t1;
# do things to t1_s and check results
drop table t1_s, t1;
drop server srv;
--disable_query_log
--disable_result_log
--source ../../t/test_deinit.inc
--enable_result_log
--enable_query_log
Please update the test accordingly.
See also inline comments along these lines, mainly removal of redundant lines.
| --echo | ||
| --echo drop and create databases | ||
| --connection master_1 | ||
| DROP DATABASE IF EXISTS auto_test_local; | ||
| CREATE DATABASE auto_test_local; | ||
| USE auto_test_local; | ||
| --connection child2_1 | ||
| DROP DATABASE IF EXISTS auto_test_remote; | ||
| CREATE DATABASE auto_test_remote; | ||
| USE auto_test_remote; | ||
| --connection child2_2 | ||
| DROP DATABASE IF EXISTS auto_test_remote2; | ||
| CREATE DATABASE auto_test_remote2; | ||
| USE auto_test_remote2; | ||
| --enable_warnings | ||
|
|
||
| --echo | ||
| --echo create child tables | ||
| --disable_query_log | ||
| --disable_result_log | ||
| --connection child2_2 | ||
| --disable_warnings | ||
| DROP TABLE IF EXISTS ta_r3; | ||
| --enable_warnings |
| --connection child2_1 | ||
| --disable_warnings | ||
| DROP TABLE IF EXISTS ta_r2; | ||
| --enable_warnings |
| SET @old_log_output = @@global.log_output; | ||
| SET GLOBAL log_output = 'TABLE,FILE'; | ||
| SET @general_log_backup = @@global.general_log; | ||
| SET @@global.general_log = 1; | ||
| TRUNCATE TABLE mysql.general_log; |
There was a problem hiding this comment.
With a single server, only one block setting query log related variables should be sufficient. Remove this
| --echo | ||
| --echo create spider table on master | ||
| --connection master_1 | ||
| --disable_query_log | ||
| echo CREATE TABLE ta_l2 ( | ||
| id INT, | ||
| val_int BIGINT UNSIGNED, | ||
| val_bin VARBINARY(4), | ||
| PRIMARY KEY(id) | ||
| ) MASTER_1_ENGINE MASTER_1_COMMENT2_P_2_1; |
| PARTITION pt1 VALUES LESS THAN (4) COMMENT='srv "s_2_1", table "ta_r2", priority "1000"', | ||
| PARTITION pt2 VALUES LESS THAN MAXVALUE COMMENT='srv "s_2_2", priority "1000001"' | ||
| ); | ||
| --disable_ps_protocol |
There was a problem hiding this comment.
Have you checked if it is necessary to disable ps protocol here? Same with ps2 protocols below.
| (4, 0x0000FF0000000000, x'0000FF00'), | ||
| (5, 0x000000FF000000FF, x'000000FF'); | ||
| --enable_ps_protocol | ||
| --enable_query_log |
| --echo | ||
| --echo deinit | ||
| --disable_warnings | ||
| --connection master_1 | ||
| DROP DATABASE IF EXISTS auto_test_local; | ||
| --connection child2_1 | ||
| SET @@global.general_log = @general_log_backup; | ||
| SET GLOBAL log_output = @old_log_output; | ||
| DROP DATABASE IF EXISTS auto_test_remote; | ||
| --connection child2_2 | ||
| SET @@global.general_log = @general_log_backup; | ||
| SET GLOBAL log_output = @old_log_output; | ||
| DROP DATABASE IF EXISTS auto_test_remote2; |
| --enable_warnings | ||
| --echo | ||
| --echo end of test |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request implements binary string support for bitwise operators and aggregate functions (MDEV-10526), including window function and Spider pushdown support. The review identified several critical issues: heap buffer overflows in Item_sum_bit due to incorrect byte allocation sizes with thd->alloc and thd->calloc, an out-of-bounds read and incorrect parsing of 64-bit unsigned integers in Spider's spd_db_conn.cc using my_strtoll10, and potential null pointer dereferences in the binary accumulator reset functions due to unchecked allocation results.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| { | ||
| if (item->m_binary_bit_counters) | ||
| { | ||
| m_binary_bit_counters= thd->alloc<uint32>(m_binary_length * 8); |
There was a problem hiding this comment.
The call to thd->alloc<uint32>(m_binary_length * 8) allocates m_binary_length * 8 bytes rather than elements because alloc takes the size in bytes. This leads to a severe buffer overflow when memcpy copies m_binary_length * 8 * sizeof(uint32) bytes into it. It should be changed to allocate the correct size in bytes.
m_binary_bit_counters= (uint32*) thd->alloc(m_binary_length * 8 * sizeof(uint32));| { | ||
| if (m_binary_mode) | ||
| { | ||
| m_binary_bit_counters= thd->calloc<uint32>(m_binary_length * 8); |
There was a problem hiding this comment.
The call to thd->calloc<uint32>(m_binary_length * 8) allocates m_binary_length * 8 bytes instead of elements. This leads to a severe buffer overflow when writing up to m_binary_length * 8 elements of uint32. It should be changed to allocate the correct size in bytes.
m_binary_bit_counters= (uint32*) thd->calloc(m_binary_length * 8 * sizeof(uint32));| int conv_error= 0; | ||
| ulonglong bits= (ulonglong) my_strtoll10(tmp_str.ptr(), | ||
| (char **) NULL, | ||
| &conv_error); |
There was a problem hiding this comment.
Using my_strtoll10 to parse unsigned 64-bit integers is incorrect because it clamps values greater than LONGLONG_MAX to LONGLONG_MAX and sets an error. Additionally, my_strtoll10 does not take a length parameter and expects a null-terminated string, which can lead to an out-of-bounds read if tmp_str is not null-terminated. We should use my_strntoull10 with tmp_str.length() instead.
int conv_error= 0;
const char *endptr;
ulonglong bits= my_strntoull10(tmp_str.ptr(),
tmp_str.length(),
&conv_error,
&endptr);| void Item_sum_or::reset_binary_accumulator() | ||
| { | ||
| m_str_value.alloc(m_binary_length); | ||
| m_str_value.length(m_binary_length); | ||
| m_str_value.set_charset(&my_charset_bin); | ||
| memset((char*) m_str_value.ptr(), 0x00, m_binary_length); | ||
| } |
There was a problem hiding this comment.
The return value of m_str_value.alloc(m_binary_length) is not checked. If the allocation fails (e.g., due to OOM), m_str_value.ptr() may return nullptr, leading to a null pointer dereference and crash in memset. We should check the return value of alloc() before writing to the buffer. Note that the same issue applies to Item_sum_and::reset_binary_accumulator() and Item_sum_xor::reset_binary_accumulator().
void Item_sum_or::reset_binary_accumulator()
{
if (!m_str_value.alloc(m_binary_length))
{
m_str_value.length(m_binary_length);
m_str_value.set_charset(&my_charset_bin);
memset((char*) m_str_value.ptr(), 0x00, m_binary_length);
}
}| if (m_binary_mode) | ||
| { | ||
| if (unlikely(direct_added)) | ||
| { | ||
| direct_added= FALSE; | ||
| direct_reseted_field= TRUE; | ||
| null_value= direct_sum_is_null; | ||
| if (null_value) | ||
| { | ||
| result_field->set_null(); | ||
| result_field->reset(); | ||
| } | ||
| else | ||
| { | ||
| result_field->set_notnull(); | ||
| result_field->store(direct_str_value.ptr(), direct_str_value.length(), &my_charset_bin); | ||
| } | ||
| return; | ||
| } |
There was a problem hiding this comment.
This part is not covered in the spider test, despite this bit added in the spider commit. Can you add test coverage?
| if (unlikely(direct_added)) | ||
| { | ||
| direct_added= FALSE; | ||
| direct_reseted_field= TRUE; | ||
| null_value= direct_sum_is_null; | ||
| bits= null_value ? reset_bits : direct_bits; | ||
| if (null_value) | ||
| result_field->set_null(); | ||
| else | ||
| result_field->set_notnull(); | ||
| int8store(result_field->ptr, bits); | ||
| return; | ||
| } |
There was a problem hiding this comment.
This part is not covered in the spider test, despite this bit added in the spider commit. Can you add test coverage?
| if (unlikely(direct_added || direct_reseted_field)) | ||
| { | ||
| direct_added= TRUE; | ||
| direct_reseted_field= FALSE; | ||
| } |
There was a problem hiding this comment.
This part is not covered in the spider test, despite this bit added in the spider commit. Can you add test coverage? Same for other bits added in Item_sum_bit::update_field in the spider commit - the method itself is not reached
| if (row->is_null()) | ||
| { | ||
| item_sum_bit->direct_add((const String *) NULL, TRUE); | ||
| } |
There was a problem hiding this comment.
Please add test coverage for this branch, same below
| { | ||
| char buf[MAX_FIELD_WIDTH]; | ||
| spider_string tmp_str(buf, MAX_FIELD_WIDTH, share->access_charset); | ||
| tmp_str.init_calc_mem(SPD_MID_DB_FETCH_FOR_ITEM_SUM_FUNC_3); | ||
| tmp_str.length(0); | ||
| if ((error_num = row->append_to_str(&tmp_str))) | ||
| DBUG_RETURN(error_num); | ||
| int conv_error= 0; | ||
| ulonglong bits= (ulonglong) my_strtoll10(tmp_str.ptr(), | ||
| (char **) NULL, | ||
| &conv_error); | ||
| item_sum_bit->direct_add(bits, FALSE); | ||
| } |
There was a problem hiding this comment.
This else branch looks very similar to the else branch above - can you refactor to reduce code duplication?
Draft for review. Implements byte-by-byte binary string mode for all scalar bitwise operators. Aggregate function
support (BIT_AND/BIT_OR/BIT_XOR) to follow.
Tested: